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Abstract — Spectral  analysis  is  a  popular  mathematical  tool  in 
analyzing  a  variety  of  network  and  distributed  systems.  For  a 
special  class  of  networks,  called  cluster  ensembles,  which  are 
made  of  interconnected  clusters,  we  can  characterize  those  which 
exhibit  strong  dominance  of  the  leading  eigenvalues  in  terms  of 
the  cluster  structure.  For  such  systems,  only  these  leading 
eigenvalues  and  their  corresponding  eigenvectors  will  need  to  be 
examined  in  studying  important  properties  of  the  underlying 
system.  This  paper  establishes  several  bounds  on  eigenvalue 
separation  ratios  in  terms  of  the  number  of  clusters,  their  sizes 
and  cluster  interconnection  topologies. 
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I  Interconnection  Network 


Figure  1.  Cluster  ensemble 


II.  An  Example  of  Cluster  Ensemble 
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I.  Introduction 

Spectral  analysis  is  a  widely  used  mathematical  tool  in 
analyzing  network  and  distributed  systems  (see,  e.g.,  [2]). 
Given  such  a  system,  e.g.,  a  sensor-target  measurement  system 
(see,  e.g.,  [5]),  it's  often  sufficient  to  examine  only  those 
dominating  eigenvectors  for  which  the  corresponding 
eigenvalues  are  much  larger  than  the  rest  of  eigenvalues.  That 
is,  these  dominating  eigenvectors  alone  can  reveal  important 
properties  of  the  system,  e.g.,  its  clusters  (see,  e.g.,  [5]).  From 
the  resulting  clustering  structure  one  can  detect,  for  example, 
malfunctioning  or  malicious  sensor  nodes  in  sensor  network 
applications  [5]. 

More  precisely,  for  a  given  system,  let  A  denote  the 
adjacency  matrix  of  the  graph  reflecting  the  underlying 
network  topology.  We  are  interested  in  the  eigen  structure  of 
AtA,  and  in  particular,  the  dominance  of  the  leading 
eigenvalues  of  ArA. 

In  this  paper  we  consider  a  class  of  network-based  systems, 
called  cluster  ensembles ,  which  are  collections  of  clusters 
connected  via  some  cluster  interconnection  network  as 
depicted  in  Figure  1.  We  show  that  for  some  cluster  ensembles 
with  regular  structures,  such  as  alpha-beta  cluster  ensembles 
defined  in  Section  III,  we  can  characterize  those  ensembles 
which  exhibit  strong  dominance  of  the  leading  eigenvalues  in 
terms  of  their  underlying  network  topologies.  These 
characterizations  can  provide  guidelines  in  designing  cluster 
ensembles  where  such  strong  dominance  is  desired. 


To  provide  a  concrete  example,  we  consider  a  cluster 
ensemble  composed  of  three  clusters  interconnected  via  a 
clique  network,  as  shown  in  Figure  2.  We  call  those  cluster 
nodes  which  are  part  of  the  cluster  interconnection  network  hub 
nodes.  Thus,  in  this  example,  nodes  10,  11  and  12  are  hub 
nodes. 


Cluster  Interconnection  Network 


Figure  2.  A  three-cluster  ensemble,  where  the  cluster 
interconnection  network  is  a  clique 

A  more  readable  drawing  of  the  same  cluster  ensemble  is 
depicted  Figure  3,  where  the  cluster  interconnection  network  is 
placed  in  the  center. 


III.  Alpha-Beta  Cluster  Ensemble 

We  focus  on  a  special  class  of  cluster  ensembles,  called 
alpha-beta  cluster  ensembles ,  which  satisfy  the  following 
properties: 

PI.  The  cluster  ensemble  is  composed  of  a  identical 
clusters. 
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P2.  Each  cluster  contains  one  hub  node  and  p  non-hub 
nodes.  This  means  that  the  size  of  each  cluster  is  p  + 
1. 

P3.  In  the  adjacency  matrix  C  of  each  cluster,  columns  of 
non-hub  nodes  are  orthogonal  to  each  other.  It  is 
possible  to  relax  this  orthogonality  condition  so  CTC 
goes  from  a  diagonal  matrix  to  a  diagonally  dominant 
one  as  described  in  [4]. 

The  alpha-beta  clusters  represent  a  natural  model  of 
connecting  multiple  clusters.  Figure  3  depicts  an  instance  of  an 
alpha-beta  cluster  ensemble  with  a  =  3  and  p  =  3. 


Figure  3.  Another  drawing  of  the  cluster  ensemble  in  Figure  2. 
It  is  an  alpha-beta  cluster  ensemble  with  a  =  3  and  p  =  3. 

In  what  follows,  the  eigenvalues  X  of  a  symmetric  matrix 
P  G  pnxn  are  aiwayS  orciereci  such  that  \  <  X2  <  ...  <  An  . 
When  it  is  useful  to  indicate  that  they  are  the  eigenvalues  on  a 
specific  matrix  P,  we  write  \  ( P )  <  X2  ( P )  <  . . .  <  Xn  ( P ) 
instead. 


IV.  Preview  of  an  Exemplar  Result 

To  have  a  quick  reading  on  the  work  of  this  paper,  we 
preview  an  exemplar  result  here. 

Note  that  for  an  alpha-beta  cluster  ensemble,  the  total 
number  of  nodes  is  a  (  P  +1).  We  use  m  to  denote  a  (  P  +1). 
Moreover,  we  use  p  to  denote  the  largest  in-degree  of  a  non¬ 
hub  node.  For  the  network  topology  in  Figure  3,  we  can  check 
that  nodes  2,  5  and  8  are  those  non-hub  nodes  which  have  the 
largest  in-degree  value  of  2.  Thus  in  this  case  p  =  2. 

Fet^4  be  the  mxm  adjacency  matrix  of  the  entire  ensemble. 
Then  we  can  show  the  following  (Corollary  4  of  Section  VI): 


eigenvalues  and  the  rest  of  non-zero  eigenvalues  will  increase 
by  a  factor  of  at  least  p/u.  This  means  that  when  P  is  large, 
there  are  a  dominating  leading  eigenvalues  and  the  number  of 
these  leading  eigenvalues  is  exactly  equal  to  the  number  of 
clusters  in  the  alpha-beta  cluster  ensemble.  Thus  we  have 
succeeded  in  relating  dominance  of  the  leading  eigenvalues  to 
cluster  size. 

To  establish  the  Theorem  and  other  similar  results  of  this 
paper,  we  will  carry  out  the  following  steps: 

51.  Consider  a  general  w-node,  ^-cluster  ensemble. 

52.  Partition  the  adjacency  matrix  A  of  the  cluster 
ensemble  and  introduce  the  cluster  interconnection 
submatrix  B. 

53.  Derive  bounds  for  the  eigenvalues  of  A1 A  in  terms  of 
those  of  B. 

54.  Consider  the  special  case  of  alpha-beta  cluster 
ensembles,  and  derive  bounds  for  the  eigenvalues  of  B 
in  terms  of  a  and  p. 

55.  Derive  bounds  on  the  separation  ratio  for  the  a  leading 
eigenvalues  of  an  alpha-beta  cluster  ensemble, 
including  the  result  of  the  Exemplar  Theorem  described 
above. 

In  the  rest  of  the  paper,  we  present  our  work  in  the  order  of 
these  steps. 

V.  Partitioning  AtA  and  Cluster  Interconnection 
Submatrix 

For  a  given  m- node,  ^-cluster  ensemble,  we  consider  its 
adjacency  matrix  A.  Since  the  eigenvalues  of  ATA  are 
invariant  under  similarity  transformations,  we  can  permute  the 
columns  of  A  so  that  its  last  k  columns  correspond  to  the  k  hub 
nodes.  We  express  ATA  in  a  partitioned  form: 


ata 


f  T  X ' 

yXT  B , 


where  J,  X  and  B  are  (m-k)x(m-k),  ( m-k)xk  and  kxk , 
respectively.  The  B  submatrix  is  of  particular  interest,  as  it 
reflects  the  interconnections  among  the  hub  nodes  and  those 
between  the  hub  nodes  and  other  nodes  in  their  respective 
clusters.  We  call  B  the  cluster  interconnection  submatrix.  In 
the  rest  of  this  section,  we  will  show  that  the  eigenvalues  of  B 
can  contribute  to  upper  and  lower  bounds  for  those  of  ATA 
(see  Corollary  1). 


Exemplar  Theorem. 

X 


K-aiA  A)  M 


Theorem  1.  (Cauchy’s  Interlacing  Theorem).  Fet  U  be  an 
mxm  symmetric  matrix  and  V  a  ( m-k )  x  (m-k)  principal 
submatrix  of  U,  for  1  <  k  <  m.  Then,  for  j  =  \, ... ,  m-k , 

A,(£0<A,(F)<A k+j(U) 


We  describe  some  system  implication  of  the  Theorem. 
Suppose  that  we  increase  P  ,  while  keeping  a  and  p  fixed. 
Then,  by  the  Theorem,  the  separation  ratio  of  the  a  leading 
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Proof:  See,  for  example,  page  189  of  [3]. 


Theorem  2.  Let  G  be  an  mxm  positive  semi-definite 
symmetric  matrix  partitioned  as 


G  = 


(H 

[xT 


x] 

A 


where  H is  ( m-k )  x  (m-k)  for  some  0  <  k<  m.  Then,  for  j  =  0, 

•  ••,  k-\. 


Proof:  The  stated  lower  bound  on  A,  .(G)  follows  from 

Theorem  10.1.1  of  [6].  The  stated  upper  bound  on  X m_j(G) 
can  be  derived  from  Aronszajn's  inequality  on  page  64  of  [1]. 


Corollary  3.  Suppose  that 


ata 


D 

X 

XT 

B 

where  ATA,  D  and  B  are  m  x  m ,  (m  -  k)  x  (m  -  k)  and  k  x  k 
matrices,  respectively,  and  D  is  a  diagonal  matrix 

D  =  diag(d{) A))  with  d{l)  <  ...  <  d(m~k) .  Then 

JLm_k+1(ATA)  ^  Xl(B) 

A,m_t(ArA)  </<-*> 


Proof:  The  result  follows  from  Corollary  2,  by  noting  that 
Xm_k(T)  =  d^m  k)  in  this  case. 


Corollary  1.  Suppose  that  ATA  is  an  mxm  symmetric  positive 
semi-definite  matrix  and  that  T  and  B  are  (m-k)  x  (m-k)  and  k 
x  k  sub-matrices,  respectively,  defined  by 


ata  = 


T  X 

kxt  b , 


Then  for  j  =  1,  . . .,  m-k , 

and  for  j  =  0,  ...,  k-1, 

^m<K,xATA)<xt_m+KJT) 


Proof:  The  first  inequality  follows  from  Theorem  1.  The 
second  inequality  follows  from  Theorem  2. 


Corollary  2.  Suppose  that 


ata 


X  J  ^ 

Ar  *) 


where  ATA,  T and  B  are  m  x  m,  (m  -  k)  x  (m  -  k)  and  k  x  k, 
respectively.  Then 


K,-kJATA)^  ISB) 
Am_k(ATA)  ln_k(T) 


Proof:  By  letting  j  =  m-k  in  the  first  inequalities  expression 
of  Corollary  1 ,  we  have 

Xm_k{ATA)<Xm_k(T) 

By  letting  j  =  k  - 1  in  the  second  inequalities  expression  of 
Corollary  1 ,  we  have 

\{B)<Xm_k+l{ATA) 

Corollary  2  follows  from  the  above  two  inequalities. 


VI.  Alpha-Beta  Cluster  Ensembles  and  Their 
Cluster  Interconnection  Networks 


To  establish  further  bounds  on  eigenvalues  of  ATA ,  we 
consider  alpha-beta  cluster  ensembles  as  defined  in  Section  III. 
Note  that  an  alpha-beta  cluster  ensemble  is  a  m- node,  ^-cluster 
ensemble  with  m  =  a(p+l)  and  k  =  a.  Let  A  be  the  adjacency 
matrix.  Then  ATA  is  a(p+l)  x  a(p+l),  and  can  be  written  as: 


ata 


"  D  JC 

Ar  A 


where  D ,  X  and  B  are  aP  x  aP,  aP  x  a,  and  a  x  a  matrices, 
respectively,  and  D  is  a  diagonal  matrix 

D  =  diag(d{l\...,d(a/]))  with  d{l)  <  ...  <  d{a(5) .  Since  each 

diagonal  element  of  D  is  the  in-degree  of  a  non-hub  node,  we 
have 


where  p  is  the  largest  in-degree  of  a  non-hub  node. 

The  cluster  interconnection  submatrix  B  reflects  the  topology 
of  the  interconnection  network  connecting  hub  nodes  of  the 
clusters.  We  consider  here  two  specific  structures,  which 
represent  two  design  extremes.  In  the  first  case,  hub  nodes  are 
densely  connected  as  a  clique,  i.e.,  every  hub  node  has  a 
directed  edge  to  every  other  hub  node,  including  itself.  In  the 
second  case,  hub  nodes  are  sparsely  connected  as  a  ring,  i.e.,  if 
the  clusters  are  labeled  as  cluster  1  through  a  then,  for  each  i  = 
1 ,  ...  a,  the  hub  node  in  cluster  i  has  a  directed  edge  to  the  hub 
node  in  cluster  (i+1)  mod  a,  and  also  a  directed  edge  to  itself. 
These  two  cases  are  depicted  in  Figure  4. 
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(a)  Clique 


(b)  Ring 


1 


Figure  4.  Two  cluster  interconnection  networks:  (a)  clique  and 
(b)  ring.  Note  that  the  alpha-beta  cluster  ensemble  of  Figure  3 
uses  the  clique  interconnection  network 


Theorem  3.  Suppose  that  the  hub  nodes  are  connected  as  a 
clique.  Then,  for  j  =  l, ...  ,a-l, 

W)=p 

and 

Xa{B)  =  a2  +  p 


2+p  1 

1  2  +  p  1 


2  +  p  1 
1  2  +  p 


By  the  Gershgorin  Disc  Theorem,  eigenvalues  of  B  are 
bounded  below  by  (2  +  j3)  —  2  and  above  by  (2  +  J3)  +  2  . 


From  Theorems  3  and  4,  we  note  that  when  the  clusters 
increase  their  size  p,  \{B)  will  increase  and  thus,  by 

Corollary  2,  also  the  eigenvalue  separation.  More  precisely, 
we  can  establish  the  corollaries  below.  . 


Corollary  4.  Suppose  that  the  hub  nodes  in  the  system  are 
connected  as  a  clique  or  ring.  Then 


K-MTa)  m 


Proof:  When  the  hub  nodes  are  connected  as  a  clique,  we 
have: 

B  =  aJa  +  pia, 

where  each  entry  of  J a  e  Raxa  equal  to  one  and 
I a  e  R axa  is  the  identity  matrix. 

Note  the  vector  u  =  (1,  1,  ...  ,  1)T  with  all  of  its  components 
equal  to  1  is  an  eigenvector  of  the  cluster  interconnection 
submatrix  B  with  the  corresponding  eigenvalue  being 

(X1  +  P  .  Let  v  be  any  other  eigenvector  of  B.  Since  v  must 
be  orthogonal  to  «,  we  have  Cv  =  0  where  C  is  a  a  x  a 
matrix  with  all  its  components  being  a.  Since  B  -C  is  a 
diagonal  matrix  with  all  its  diagonals  being  p,  we  have 
Bv  =  (2?  —  C)v  +  Cv  =  (B  —  C)v  —  f3v.  Thus,  the 

eigenvalues  of  B  are  Aj{B)  =  J3  for  j  =  —  1  and 

4  (B)  =  a2+p. 

Theorem  4.  Suppose  that  the  hub  nodes  are  connected  as  a 
ring.  Then,  for  j  =  1, ...  ,a, 

J3<Aj(B)<J3  +  4 

Proof:  When  the  hub  nodes  are  connected  as  a  ring,  we  have 


Proof:  The  result  follows  from  Corollary  3  and  Theorems  3 
and  4,  while  noting  that  d{a^  =  JU  . 

Corollary  5.  Suppose  that  the  hub  nodes  in  the  system  are 
connected  as  a  clique.  Then 

AJSA)  >a^  +  p 
K-Mta)  p+/u 

Proof:  From  Theorem  2,  we  note  that,  for 

7  =  0,  ...  ,k- 1, 

<  K-MTA)  <  \_jiB)  +  Am_k(D) 

For  j  =  0,  we  have 

Xk{B)<Xm{ATA) 

For  j  =  1,  we  have 

4_,(/ir.4)<4_l(fl)+4_t(i>) 

By  Theorem  3,  we  have 

Xk  (B)  =  a2  +  {3  and  AkA(B)  =  p 
It  follows  that 

Xm(ATA)>Xk(B)  =  a2  +  p 

and 

Xm_l(ATA)<Xk_l(B)  +  Xm_k(D)  =  p  +  Ju 

by  noting  that  X m_k  ( D )  =  JU  .  The  Corollary  follows  from  the 
above  two  inequalities. 
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Corollary  6.  Suppose  that  the  hub  nodes  in  the  system  are 
connected  as  a  ring.  Then 

Am(ATA)  ^,/?  +  4  +  // 

K-kMTA)  ~  p 

Proof:  From  Corollary  1,  we  note  that,  for 
7  =  0,  1, 

*  ^MtA)  <  +  ^(D) 

For  /  =  0,  we  have 

AJArA)<Ak(B)  +  Am_k(D ) 

For  j  =  k—  1,  we  have 

\{B)<Xm_k+x{ATA) 

By  Theorem  4,  we  have 

Ak(B)<p  +  4andAl(B)>p 

It  follows  that 

4(^)<^(5)+4_,(Z))<^+4+// 

and 

(ATA)>m>p 

The  Corollary  follows  from  the  above  two  inequalities. 

VII.  System  Implication 

We  can  design  a  cluster  ensemble  system  which  will  lead  to 
a  desired  eigenvalue  separation  ratio  of  the  leading  eigenvalues 
and  the  rest  of  the  non-zero  eigenvalues.  For  example,  we  can 
form  an  alpha-beta  cluster  ensemble  with  a  relatively  large  P, 
so  that  p/p  will  be  large,  where  p  is  the  largest  in-degree  of  a 
non-hub  node  in  a  cluster.  Then,  when  the  hub  nodes  are 
connected  as  either  a  clique  or  ring,  Corollary  4  implies  that  the 
resulting  system  will  exhibit  strong  dominance  by  the  a  leading 
eigenvalues.  This  means  that  the  overall  system  behavior  will 
essentially  be  governed  by  the  corresponding  a  eigenvectors. 
For  example,  the  values  or  signs  of  the  components  in  these 
eigenvectors  will  reveal  clustering  structures  of  the  system  [7]. 
The  resulting  clusters,  for  example,  can  help  detect 
malfunctioning  or  malicious  sensors  in  a  sensor  network 
application  [5]. 

Furthermore,  suppose  that  values  for  a  and  p  are  fixed  and 
the  hub  nodes  are  connected  as  a  ring.  Corollary  6  implies  that 
separation  ratios  among  the  largest  a  eigenvalues  will  be 
bounded  above  by  a  quantity  which  will  approach  1  when  p 
increases.  Thus,  for  large  p,  there  will  be  large  separations 
between  the  a  leading  eigenvalues  and  the  rest  of  eigenvalues 
(Corollary  4),  but  there  are  no  significant  separations  among 
the  a  leading  eigenvalues  themselves  (Corollary  6).  This  means 
that  when  the  hub  nodes  are  connected  as  ring  all  the 


corresponding  a  eigenvectors  will  be  of  equal  significance  in 
characterizing  the  system. 

As  a  contrast,  consider  the  case  when  hub  nodes  are 
connected  as  a  clique.  Assume  that  the  values  for  p  and  p  are 
fixed.  Then,  by  Corollary  5,  when  (a2  +  P)/(p  +p)  is  large  (e.g., 
when  a  is  large  relatively  to  P  and  p),  the  principal  eigenvalue 
will  exhibit  strong  dominance.  Thus,  for  example,  component 
values  of  the  principal  eigenvector  alone  will  basically 
characterize  the  clustering  structure  of  the  system  [7]. 

From  the  above  discussion,  we  see  that  by  choosing  proper 
values  for  a,  p  and  p  in  the  design  of  a  cluster  ensemble,  we 
can  tune  the  dominance  properties  of  its  leading  eigenvalues. 

VIII.  Summary  and  Concluding  Remarks 

We  have  motivated  the  problem  of  designing  distributed 
systems  which  exhibit  strong  dominance  of  the  leading 
eigenvalues,  in  terms  of  underlying  network  topologies.  We 
have  introduced  a  special  class  of  networks  called  cluster 
ensembles,  and  established  some  fundamental  mathematical 
results  of  ensuring  such  dominance  for  this  class  of  networks. 
As  future  work,  we  plan  to  generalize  these  characterizations 
by  allowing  more  general  topology  than  the  alpha-beta 
ensemble  and  develop  applications  in  system  design  based  on 
spectral  properties  established  in  this  paper. 
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